How do I extract the individual flows from the total packets in a pcap file?

asked 2018-04-04 16:31:40 +0000

yyl05
1 ●1 ●2 ●1

Hello all,

I would like to do some calculations on the network flows by using the packet fields to calculate values such as the variance of payload packet length for a time interval, number of packet exchanged for time interval and so on, like from this paper: (http://www.jatit.org/volumes/Vol67No2...)

I understand that a flow is a series of packets sharing same characteristics such as Src.address, Src.port, Dest.Addres, and Dest.port. Right now I can extract packet fields to a csv file using tshark, however it is for all packets instead of a flow. How can I extract the information such that I have the individual flows instead of the individual packets so that I can perform calculations on it?

Thank you so much for your time

edit retag flag offensive close merge delete

add a comment

2 Answers

Sort by » oldest newest most voted

answered 2018-04-04 17:27:41 +0000

Jasper
24103 ●5 ●111 ●292 Düsseldorf, Germany

What I would do is to run tshark with a display filter (using the -Y parameter) that limits the packets for which values are exported. You could either use a tuple filter, specifying both IPs and ports, or simply use the stream index filter, e.g.

 tcp.stream==1

edit flag offensive delete link

Comments

Thank you!

yyl05 ( 2018-04-05 07:24:45 +0000 )edit

add a comment

answered 2018-04-04 17:51:39 +0000

cmaynard

11115 ●11 ●317 ●165 https://www.igt.com/

If you're trying to process multiple flows, then you will most likely need to script something, and as Jasper alluded to, the stream number can be used as the key to doing this; however, first you have to find out how many streams there are.

For convenience, I'm pasting a script that Sake originally provided in his answer to the question, "Easy way to save tcp streams?" over at the old Q&A site, along with my follow-up for Windows:

for stream in `tshark -r <pcapfile> -T fields -e tcp.stream | sort -n | uniq`
do
    echo $stream
    tshark -r <pcapfile> -w stream-$stream.cap -R "tcp.stream==$stream"
done

If you're using Cygwin on the Windows platform, you may need to pipe the output of uniq to sed to remove the extraneous carriage return; otherwise you may see an invalid address:port pair error message, i.e.:

for stream in `tshark -r <pcapfile> -T fields -e tcp.stream | sort -n | uniq | sed 's/\r//'`

Obviously, you may need to modify the tshark command to suit your exact needs.

edit flag offensive delete link

Comments

Thank you, i will look into it! Also, I would like to ask if this script is to be run on linux command line? Sorry if the question is too basic as I am very new to this. Thank you so much for your help

yyl05 ( 2018-04-05 07:59:24 +0000 )edit

The script was written for *nix, but can be run in Cygwin with modifications as indicated.

But after looking at "Fig. 6: Network Flow Attributes" from the linked FLOW BASED ANALYSIS TO IDENTIFY BOTNET INFECTED SYSTEMS paper, maybe what you're seeking can be more easily obtained by running something like this instead?

tshark -r file.pcap -z conv,tcp

Refer to the tshark man page for more information.

cmaynard ( 2018-04-05 17:44:46 +0000 )edit

Great I will try it out, thanks alot! Is there also a way for tshark to have the statistics for a time interval? Example have the statistics for every 300seconds?

yyl05 ( 2018-04-07 02:00:51 +0000 )edit

Maybe have a look at the -z io,stat,interval option? If you want statistics by stream then you can modify the script above slightly like so:

for stream in `tshark -r <pcapfile> -T fields -e tcp.stream | sort -n | uniq`
do
    echo $stream
    tshark -r <pcapfile> -z io,stat,300,"tcp.stream==$stream"
done

... or leave the read filter if you wish and don't bother with the filter for the -z io,stat,interval option.

cmaynard ( 2018-04-07 14:59:38 +0000 )edit

add a comment